The present invention relates to multimedia systems, and more particularly to a subliminally embedded key in a video signal for synchronization of playout control of multimedia data and/or of execution of instructions contained in control data.
Watermarking of video signals is becoming of increasing use as a means of transmitting very low rate data as part of the video signal. Watermarking of video has been proposed for performing audio to video delay compensation, for indicating authentication or program ownership and for verifying playing of a video signal such as in advertisement logging. The watermarking technique inserts the very low rate data into the active portion of the video signal so as to be invisible while still having the robustness to survive video signal processing including multiple generations of compression encoding/decoding. See U.S. Pat. No. 4,969,041 issued to William O""Grady et al on Nov. 6, 1990 entitled xe2x80x9cEmbedment of Data in a Video Signalxe2x80x9d and co-pending U.S. patent application Ser. No. 08/829,524 by Daniel Baker et al filed on Mar. 28, 1997 entitled xe2x80x9cTransparent Embedment of Data in a Video Signalxe2x80x9d.
Also the use of auxiliary channels for transmitting data related to video signals is known, as shown in U.S. Pat. No. 6,075,561 issued to Bozidar Janko on Jun. 13, 2000 entitled xe2x80x9cLow Duty-Cycle Transport of Video Reference Imagesxe2x80x9d. With the advent of the Internet, proposals have been made to use the Internet as the auxiliary channel for transporting data. It is anticipated that when video that is streamed over the Internet is played out, it may be desirable to also play out other dataxe2x80x94called metadataxe2x80x94that is associated with the video to immerse the viewer in the experience. It may also be desirable to transmit control data to the equipment receiving the video. This control data may be used for one or more of the following purposes: (1) cue a commercial; (2) specify a destination to which the video is to be routed; (3) control auxiliary equipment to actuate other sensory cues such as activation of special lighting or sound effects; (4) control the parameters for compressing the video prior to storage; etc.
However transmitting the metadata is not currently possible using watermarking technology with video. The bit rate of the metadata, such as 9600 baud, is too high for the very low data rate channel afforded by watermarking. Transmission of control data may or may not be possible using watermarking technology directly, depending upon the quantity of data which varies with the application.
The problem that arises is how to synchronize the metadata or other control data with the video when they are transmitted over different channels so that the metadata appears at the proper point with relationship to the video when seen by the viewer, or the control data is retrieved and used at the proper point with relationship to the video.
What is desired is a means for synchronizing the metadata or control data with the associated video when the video is played out.
Accordingly the present invention provides subliminally embedded keys in video that act as pointers and synchronizing signals for a data file containing metadata and/or control data so that the metadata is displayed with the video and/or instructions contained in the control data are executed at the proper time. At the transmission end of a system a key is generated in synchronization with the video, the key being associated with corresponding data file. The key is embedded subliminally in the video at an appropriate point using watermarking techniques. At a receiver the key is decoded from the video and used to access a network server that contains the data file. Once retrieved the metadata from the data file is displayed with the video at a point in synchronization with the video as indicated by the key, or the control data is used to perform one or more auxiliary functions at designated times. For small data files where the retrieval delay is small, the key may be just a few frames before the first video frame which is associated with the data file, while for large data files the key may be inserted several seconds before the first video frame which is associated with the data file. The key provides both an xe2x80x9caddressxe2x80x9d for retrieving the data file and a time from the key frame when the metadata from the data file is to be played out with the video for a viewer or when the instructions contained in the control data are to be executed. In this way the data file is synchronized with the video.
The objects, advantages and other novel features of the present invention are apparent from the following detailed description when read in conjunction with the appended claims and attached drawing.